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I  DUnTF LABILITY  OF  MIXTURES  OF  EXPONENTIAL  FAMILIES 

by 

0.  Bamdorff -Nielsen 

Let  -  {F( *  | t)  s  t  €  T}  be  a  family  of  n- dimensional  dis¬ 
tribution  functions  (d.f , s . )  depending  on  a  o- dimensional  parameter  t 
which  ranges  over  a  Borel  set  T  in  Rm,  the  m-dlmensional  Euclidian 
space.  We  assume  that  for  each  fixed  x  =  (x^,  ...  ,  xq)  e  Rn  the 
function  F(x|‘)  is  Borel  measurable.  Let  denote  the  set  of 

all  probability  measures  (p.m.s.)  on  the  Borel  field  fin  of  Rn 
(®m  of  Rm)  and  let  A  denote  the  set  of  those  7  ejf  for  which 
7(T)  =  1.  Hie  family  determines  a  mapping  i|r  :  by  the 

relation 

(1)  *(7)  =  /  F(*  |t)  d7(T) 

T 

We  speak  of  the  d.f.  t(7)  as  a  mixture  of  5^  (w.r.t.  7).  Hie  mapping 

t  is  said  to  be  identifiable  if  it  is  one  to  one.  In  certain  connections 
(e.g.  statistical  estimation  of  7)  it  is  important  to  know  whether  f 
is  identifiable.  Various  conditions  for  identif lability  and  nonldenti- 
f lability  are  known,  see  Teicher  [4]  and  the  references  therein.  Here 
we  want  to  prove  that,  under  mild  restrictions,  mixtures  of  exponential 
families  are  identifiable.  is  exponential  (or  of  the  Darmois- 
Koopman  type)  if  for  some  o-finite  measure  u 


(2) 


C  hj(x) 

dF(x|x)  -  a(x)  b(x)  e^"1  dji(x) 

for  x  e  Rn,  t  ■  t2>  ...  ,  x  )  €  T,  where  a(r)  >  0,  b(x)  >  0 

and  a,  b,  hj,  J  =  1,  ...  ,  m  are  all  measurable. 

Let  7X>  r2  e  and  let 

d*(7v)  5  TJ  hJ(x) 

(3)  fy(x)  *  -  b(x)  /  a(x)  e1  d7v(T),  v  =  1,  2. 

Furthermore,  let  4  =  {x  :  f^x)  =  fg(x)  i  0),  let 
T]  =  {y  =  (hx(x) ,  ...  ,  hB(x))  :  x  e  {}  and  let 

(4)  f*(y)  =  /  a(t)  e(x'y)  d7  (t),  v  =  1,  2. 

v  T 

where  (t,  y)  denotes  the  inner  product  of  x(£  T)  and  y  e  R  .  Then 
*  # 

f-^y)  =  f 2 (y)  if  y  e  ij;  0UT  aim  is  show  that  under  certain  further 
restrictions  this  implies  7^  =*  7g.  Let  c(tj)  denote  the  convex  hull 
of  tj.  We  shall  distinguish  between  four  cases. 

(i)  t)  is  finite. 

(ii)  tj  is  infinite,  c(tj)  Is  bounded  and  tj  does  not  have  an  accu¬ 
mulation  point  in  the  interior  of  c(tj). 

(iii)  As  (il)  except  that  c(tj)  is  assumed  unbounded. 
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(iv)  i]  is  infinite  and  t)  has  an  accumulation  point  in  the  interior 
of  c(r}). 

Case  (i).  The  important  example  of  this  case  is  the  binomial  distri¬ 
bution.  An  analysis  of  the  identif lability  problem  for  that  distribution 
can  be  found  in  [4]. 

Case  (li).  From  the  viewpoint  of  statistics  (ii)  is  the  case  of  least 
interest.  We  have  obtained  no  general  results.  The  problem  is  essen¬ 
tially  this:  (n  =  m  =  1).  Let  y^  and  y 2  be  two  p.m.  's  on  (R ,(8) 
whose  Laplace  transforms  q>^( z )  and  cp2(z)  both  exist  in  a  strip 
0  <  f\e  z  <  p,  p  >  0.  Let  (xn)  be  a  sequence  of  real  numbers  such 
that  0  <  xn  <  p  for  all  n  and  xn  -*  0  as  n  -*  » .  Find  conditions 

under  which  q>^(xn)  =  <J>2(xn)  ^or  n  imP^ies  =  <J>2(it)  for 

all  real  t  (i.e.,  identity  of  the  Fourier  transforms  of  7^  and  y 2 
and  hence  identity  of  y ^  and  y^) . 

Case  (lii).  We  shall  treat  the  subcase: 

(iii) 1 .  i)  contains  the  set  I+  of  all  lattice  points  in  Rm  with 

nonnegative  components,  i.e.,  I+  =  {k  =  (k.. ,  . . .  ,  k  )  :  k .  is  a  non- 

x  m  j 

negative  integer,  J  =  1,  ...  ,  m). 

We  have,  since  0  =  (0,  ...  ,  0)  e  I+ 

(5)  f*(0)  =  /  a(t)  d7,(T)  =  /  a(x)  dr2(T)  =  fp(0), 

T  T 
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Let  us  denote  the  common  (positive)  value  in  (5)  by  c  and  let  us 

*  *  -1 

introduce  the  p.m. 's  yv,  v  =  1,  2,  by  dy^r)  =  c  a(x)  d7v(r).  Thus 


(6)  f*(k)  =  /  e(r’k)  dr*(t)  =  /  e(T’k)d72(T)  =  f!(k)  V  k  e  I+  . 

T  T 


Let  v  be  the  transformation  :  x  -»  X  =  w(r)  where  X  =  (X. ,  ...  ,  Xm) 
=  (e  ,  ...  ,  e  let  A  =  w(T)  and  =  7  w“  ,  v  ■  1,  2.  We 
obtain  from  (6) 


d*2(X) 


V  k  -  (k^,  ...  f  k^)  €  I  • 


We  can  draw  the  following  conclusion. 

Proposition  1.  Suppose  that  assumption  (iii)'  is  satisfied  and  suppose 
that  and  «2  are  uniquely  determined  by  their  moments  (7).  Then 

K1  =  n2  an<*  c0115®^11®11^!/  7 2.  =  7 2’ 

In  order  to  derive  a  sufficient  condition  for  7^  =  7 2  which 
is  more  useful  than  that  of  Proposition  1  we  state  the  following  lemma. 

Lemma  1.  Let  n  be  an  arbitrary  p.m.  on  (Rm,  fim)  with  jtfR*111)  =  1 
where  R+  is  the  set  of  nonnegative  reals  and  with  all  moments 

(8)  =  f  h1  dn(X),  k  e  I+ 

1  m  'J 


k 


finite.  If  there  exists  a  positive  number  p  such  that  the  series 


(9) 


k  e  I 


kl  + 


kl> 


+  k 


k  ! 
m 


is  convergent  then  n  is  the  unique  p.m.  with  these  moments. 

The  lemma  and.  its  proof  are  straightforward  generalizations 
of  a  result  in  the  book  of  Cramer  [2;  176]. 

Let  us  apply  the  lemma  to  (7)-  We  find  (dropping  the  sub¬ 
script  v) 


n  m  /  "  (X.  p)  s 

■/  n  (&  «  )d" 

A  j=l 

=  i  f  a(-r)  ep  Z  e  J  d'/(t) 


-  1  /  ,  p  £  e. 

<  -  sup  a(t)  e 

"  °  t  e  T  • 


Therefore 
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Proposition  2.  Suppose  that  assumption  (ill)'  is  satisfied  and  suppose 


that 


(10) 


sup  a(r) 
t  e  T 


,P  £  e 


TJ 


<  00 


for  some  p  >  0.  Then  y ^  =  y 

As  an  application,  let  us  consider  the  instance  where  n  =  m, 
h^(x)  =  Xj  (j-th  coordinate  of  xj  J  =  1,  2,  ...  ,  m)  and  where  the 
measure  4  in  (2)  is  concentrated  on  I+;  then  without  loss  of  gener¬ 
ality  we  can  and  will  assume  4  to  be  counting  measure  on  I+.  Hence 
the  family  is  given  by 


[£] 


1  2  s(t)  b(k)  e 
k=0 


(11) 


F(x|t)  = 


(T,k) 


if  x  >  0 


otherwise 


in  an  obvious  notation.  Assumption  (iii)'  becomes:  b(k)  >  oVk  e  I+ 
and  we  have 


Corollary  1.  If  the  family  9^  given  by  (11)  satisfies  b(k)  >  O^k  e  I+ 
and 

TJ 

(12)  sup  a(r)  eP  ^  6  <  » 

t  e  T 

for  some  p  >  0  then  \(f  is  identifiable. 

Specializing  still  further  we  obtain  (Feller  [5]) 
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Corollary  2.  The  mapping  \|r  determined  by  the  Poisson  family 


=  {F( '  | t)  :  -»  <  t  <  where 


F(x|t) 


CE] 

k=0 


is  identifiable. 


x  >  0,  X  =  eT 


Case  (lv).  We  shall  prove  that  7 1  =  7 2  provided 

(iv)’.  There  exists  an  accumulation  point  y^°^  =  (y!°^,  ...  ,  y^) 
"  ■  j.  m 

of  n  in  the  interior  of  c(tj)  with  the  following  property.  If  two 
arbitrary  complex  power  series 


£  a 


(v) 

J2 


<*,  -  y[o))  1  (z, 


“m 


y<0))J2 


(z, 


m 


v  =  1,  2 


coincide  for  all  z  =  (z^,  ...  ,  z^)  e  t)  H  V  for  some  neighborhood  V 
of  then  they  have  identical  coefficients. 

We  note  that  assumption  (iv)’  is  equal  to  (iv)  if  m  =  1.  A 
sufficient  condition  for  (iv)’  is  that  rj'  be  dense  in  some  open  subset 
of  Rm. 


Proposition  3.  Suppose  that  assumption  (iv)'  is  satisfied.  Then 
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Proof.  Without  loss  of  generality  we  can  and  will  assume  that  the 


origin  0  is  in  and  that  there  is  a  neighborhood 
K  =  {y  :  |y^  |  <  p,  j  =  1,  . . .  ,  m)  of  0  for  which  K  c  c(rj)  and  K 
contains  y^.  Then 

(13).  f*(0)  =  /  a(T)  dy  (t)  =  /  a(x)  dy2(t)  =  f*(o)  . 

T  T 

Let  us  denote  the  common  value  in  (13)  by  c  and  let  us  define  the 
*  X 

p.m.s.  v  =  1,  2  by  d7*(t)  =  i  a(r)  d 7v(t).  Furthermore,  let 

<Pv,  v  =  1,  2  denote  the  Laplace  transform  of  y 

<Pv(z)  =  /  e^T,z)  iy*(t) 

T 

where  z  =  (z.. ,  ...  ,  z  ),  z.  =  u.  +  iv,  (j  =  1,  ...  ,  m).  <p  exists 
1  m  '  J  J  J  v 

for  all  z  e  K'  =  (zju  =  (u^,  ...  ,  u^)  e  K) .  In  fact,  for  any  such 
z,  jexp((x,  z))|  <  exp((x,  u))  and  a  moments  reflection  shows  that 
there  exists  a  y  c  ij  with  (t,  u)  <  (t,  y);  thus 

/  |e^T,z^ |d7*(t)  <  |  /  a(x)  e^T,y^  dy  (t)  <  » . 

T  ”  T 

More  is  true:  <j>v  is  an  analytic  function  of  z  =  (z^,  ...  ,  zm)  in 
the  domain  K'.  To  prove  this  it  suffices  to  show  that  q>v  is  analytic 
in  each  of  the  variables  Zy  j  =  1,  ...  ,  m  (see  [1]).  Hence  let 
us  consider 
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(14) 


(p  (z  +  he.)  -  <p  (z) 

- - ~t - — 


z)  e 


T,h 


1  *,  V 

-  %(*) 


where  z=u+iv£K',e.  denotes  the  j-th  unit  vector  in  Rm  and 

u 

h  is  an  arbitrary  conplex  number.  Let  6  >  0  be  so  small  that 

z  +  he.  e  K'  for  all  h  such  that  |h|  <8.  Using  the  (well  known) 
J 

inequality 


1 


for  |h |  <  8 


we  find  that  the  integrand  in  (14)  is  dominated  by 


■,  /  (t,  u  +  8  e  )  (t,  u  -  6  e  )  v 

K*  ) 


and  since  the  integral  of  this  quantity  is  finite  we  nay  pass  to  the 
limit  h  -*  0  under  the  integration  sign  in  (14)  to  obtain 


<p(z 


<PU) 


/,J 


.(l,z) 


<*7„(t) 


as  h  -*  0 


We  have  thus  shown  that  <pv  is  analytic  in  K1.  Consequently  <py  can 
be  expanded  in  a  power  series  around  z^°^  =  y^°^ 


<pvc*) 


Z  a 


(v) 

J1  J2 


J  ^Z1 

Jm 


<*2  -  40))Ja 


(z„ 


y(o))3« 
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the  expansion  feeing  valid  in  some  neighborhood  V  of  y^°\  We  have 
cpj.(z)  =  <P2(z)  V  z  £  t)  and  hehce,  fey  assumption  (iv)'  and  uniqueness 
of  analytic  continuation,  cp^(z)  =  cp2(z)  ^  z  e  K' .  In  particular 

<p^( z)  =  tp2(z)  for  all  purely  imaginary  z  =  iv  =  (iv^  ...  ,  iv^),  i.e., 

*  *  *  * 
the  characteristic  functions  of  7^  and  7^  coincide,  hence  7^  =  7 2 

or,  equivalently,  7^  =  7 2>  q.e.d. 

By  the  remark  preceding  Proposition  3>  ve  obtain 


Corollary  3-  Suppose  that  in  the  representation  (2):  (a)  n  is 

n-dimenslonal  Lebesgue  measure,  (fe)  the  functions  h  ,  j  =  1,  ...  ,  m 

J 

are  all  continuous,  (c)  the  set  (y  :  y  =  (h^Cx),  ...  ,  hm(x)), 
b(x)  >  0,  x  e  Rn}  contains  a  (nonempty)  open  set.  Then  if  is 
identifiable. 

Specializing  still  further  we  get 


Corollary  4.  Suppose  that  is  the  Gaussian  family 


(15) 


®(x1,  •••  ,  ^1^,  Tg) 


TX  < 

0 

A 

ro 

A 

“)> 

n 

II 

ro 

a 

) 

) 

'  2 

exp 

(-+, 

c  <v*)2) 

V  20 

f=i  ^  / 

/  T  2  N 

n 

(2 

/ 

2 

n  T1  \ 

h-L(x)T1+h2(x)  t2 

=  V  2w  / 

r 

exp  f  - 

-2  -2 

where  u  is  n-dimensional  Lebesgue  measure,  =  (  0  ,  r2  =  o  , 
h^(x)  =  E  and  h2(x)  =  -  ^  Z  x^.  If  n  >  1,  then  if  is  identifiable 
(Teicher  has  shown,  see  [5],  that  f  is  not  identifiable  if  n  =  1). 
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